Contraction mapping of feature norms for data quality imbalance learning

Abstract

The popular softmax loss and its recent extensions have achieved great success in deep learning-based image classification. However, the data for training image classifiers often exhibit a highly skewed distribution in quality, i.e., the number of data with good quality is much more than that with low quality. If this problem is ignored, low-quality data are hard to classify correctly. In this paper, we discover the positive correlation between the quality of an image and its feature norm ($L_2$-norm) learned from softmax loss through careful experiments on various applications with different deep neural networks. Based on this finding, we propose a contraction mapping function to compress the range of feature norms of training images according to their quality and embed this contraction mapping function into softmax loss and its extensions to produce novel learning objectives. Experiments on various applications, including handwritten digit recognition, lung nodule classification, and face recognition, demonstrate that the proposed approach is promising to effectively deal with the problem of learning quality imbalance data and leads to significant and stable improvements in the classification accuracy. The code is available at https://github.com/Huiyu-Li/CM-M-Softmax-Loss.