Few-Shot Semantic Segmentation Based on Dual-Branch Feature Extraction

Abstract

Few-shot semantic segmentation (FSS) requires only few labeled samples to achieve good segmentation performance and thus has received extensive attention. However, existing FFS methods usually adopt a simple convolutional structure as the backbone, which suffers from poor feature extraction ability. In order to address this issue, a novel few-shot segmentation network based on dual-branch feature extraction (DFESN) is proposed. First, an attention-enhanced ResNet is used as the local feature extraction branch. Specifically, we in-corporate channel attention operations into each building block of ResNet to model the importance among channels, which enables DFESN to learn important class information for the segmentation task. Besides, we introduce a Vision Transformer as the global feature extraction branch. This branch leverages the multi-head self-attention mechanism in Vision Transformer to model the global dependencies of support and query image features, further enhancing the feature extraction capabilities of DFESN. We conduct experiments on the PASCAL-5i dataset and demonstrate the superiority of our DFESN.