AsPrompt: Attribute-structured knowledge-guided dual-modal coupling prompt learning for few-shot image classification

Abstract

The few-shot image classification task involves classifying images when only a limited number of training images are available. This field has seen significant advancements in recent years due to the development of pre-trained vision-language models (e.g., CLIP), which exhibit strong generalization capabilities. Recent studies have further leveraged classes-related descriptions as part of prompt learning to better adapt these foundational vision-language models for downstream tasks. However, the textual descriptions used in traditional methods often lack sufficient class-discriminative information, limiting the model’s expressiveness on unseen data domains. Given that large language models possess rich structured knowledge bases, they offer new avenues for enhancing textual information. Against this backdrop, we propose a novel method called AsPrompt, which integrates attribute-structured knowledge guidance with a dual-modal coupling prompt learning mechanism. This approach not only enriches class-discriminative textual information but also effectively integrates structured knowledge with traditional textual information by capturing the structured relationships between entity sets and attribute sets. Experimental results demonstrate that AsPrompt surpasses other state-of-the-art prompt learning methods on 11 different few-shot image classification datasets, showcasing its superior performance. The code can be found at https://github.com/SandyPrompt/AsPrompt.