Learning Neural Network Classifiers by Distributing Nearest Neighbors on Adaptive Hypersphere

Abstract

In this study, the adaptive hypersphere nearest neighbors (ASNN) method is proposed as an optimization framework to enhance the generalization performance of neural network classifiers. In terms of the classification task, the neural network draws decision boundaries by constructing the discriminative features of samples. To learn those features, attributed to the flexibility and separability, the pair-wise constraint-based methods that consist of the pair-wise loss and an embedding space (e.g., hypersphere space) have gained considerable attention over the last decade. Despite their success, pair-wise constraint-based methods still suffer from premature convergence or divergence problems, driven by two main challenges. 1) The poor scalability of the embedding space constrains the variety of the distribution of embedded samples, thereby increasing the optimization difficulty. 2) It is hard to select suitable positive/negative pairs during the training. In order to address the aforementioned problems, we propose an adaptive hypersphere nearest neighbors method. On the one hand, we improve the scalability of features via a scale-adaptive hypersphere embedding space. On the other hand, we introduce a neighborhood-based probability loss, which magnifies the difference between pairs and enhances the discriminative power of features generated by the neural networks based on the nearest neighbor-based pairing strategy. Experiments on UCI datasets and image recognition tasks demonstrate that the proposed ASNN not only achieves improved intraclass consistency and interclass separability of samples, but also outperforms its competitive counterparts.