Saccade and purify: Task adapted multi-view feature calibration network for few shot learning

Abstract

Current few-shot image classification methods encounter challenges in extracting multi-view features that can complement each other and selecting optimal features for classification in a specific task. To address this problem, we propose a novel Task-adapted Multi-view feature Calibration Network (TMCN) inspired by the different saccade patterns observed in the human visual system. The TMCN is designed to “saccade” for extracting complementary multi-view features and “purify” multi-view features in a task-adapted manner. To capture more representative features, we propose a multi-view feature extraction method that simulates the voluntary saccades and scanning saccades in the human visual system, which generates global, local grid, and randomly sampled multi-view features. To purify and obtain the most appropriate features, we employ a global local feature calibration module to calibrate global and local grid features for achieving more stable non-local image features. Furthermore, a sampling feature fusion method is proposed to fuse the randomly sampled features from classes to obtain better prototypes, and a multi-view feature calibrating module is proposed to adaptively fuse purified multi-view features based on the task information obtained from the task feature extracting module. Extensive experiments conducted on three widely used public datasets prove that our proposed TMCN can achieve excellent performance and surpass state-of-the-art methods. The code is available at the following address: https://github.com/huyunzuo/TMCN.