SGE: Semantic-guided Generalization Enhancement for Few-Shot Learning

Abstract

In the field of computer vision, learning from limited datasets presents a significant challenge, known as Few-Shot Learning (FSL). Current methodologies have employed semantic information and data augmentation to address insufficient feature representation in limited datasets. However, these approaches often fail to resolve the fundamental data scarcity problem. Additionally, they may inadvertently introduce distributional bias, constraining the utility of semantic features in FSL contexts. To alleviate the issue, in this paper we propose a novel Semantic-guided Generalization Enhancement method (SGE) for FSL. SGE masterfully harnesses robust semantic information by integrating knowledge from diverse pre-trained models. It utilizes semantic cues from class labels to guide both data augmentation and feature extraction processes. Compared to traditional data augmentation techniques, SGE generates augmented samples that are more semantically consistent with the original samples. SGE enables the model to obtain a comprehensive and accurate representation of class characteristics through multifaceted data augmentation of categories. The backbone model adaptively integrates semantic information with image data, and merges the enhanced sample features with the original sample features via a feature fusion module. This allows SGE, guided by semantic cues, to construct a robust class prototype with rich discriminative features. Empirical evidence demonstrates that our framework outperforms state-of-the-art methods across four benchmark assessments. This clearly demonstrates the remarkable efficacy of SGE in leveraging semantic information, thereby exerting a significant influence on FSL. The code is available at https://github.com/zhuyangyang-cjlu/SGE.