Reducing hubness to improve inductive few-shot learning

Few-Shot Learning (FSL) provides a promising paradigm that completes a task using a few labeled samples, benefiting object recognition, outlier detection, and various tasks. However, distance-based FSL is vulnerable to the hubness problem, i.e., a few points (hubs from one class) occur frequently among the nearest neighbors of other points (from other classes) in the high-dimensional space. Therefore, the hubs commonly mislead classifiers resulting in a considerable performance degradation. The major body of current solutions is not designed to approach zero hubness, or limited to the transductive paradigm. As a countermeasure, this work aims to approach zero hubness for inductive FSL explicitly. We propose a new optimization objective that restricts the embeddings to approach zero hubness. A holistic Hubness rEDucing (HED) method is proposed to embed representations on the hypersphere and maintain linear separability, resulting in the decrease of hubness and preservation of class structure. A calibration mechanism is further devised to mitigate the negative impact of FSL’s data limitation, which calibrates the potentially biased distribution of embeddings. Extensive evaluation results demonstrate that our proposed method reduces hubness to improve the inductive FSL, and meanwhile, is compatible with a wide range of backbones.