Prototype optimization and self-training for few-shot 3D point cloud semantic segmentation

Abstract

Few-shot point cloud segmentation aims to accurately decompose 3D point clouds into different semantic classes with few samples, and is crucial for subsequent tasks, such as analysis, modeling and editing. Despite the popularity of prototype-based approaches, prototypes often fail to adequately capture class-specific information. Therefore, for each class, a few points may exhibit significant differences from their prototype. And the lack of sufficient distinction between foreground and background prototypes presents a great challenge for precise segmentation. To address these issues, we propose a prototype optimization module to mitigate the interference among support prototypes, thereby generating prototypes of superior quality. These refined prototypes are capable of capturing the key characteristics of the data, which can prominently improve the generalization capability of our model. Then, we devise a self-training strategy that leverages pseudo query prototypes generated from high-confidence predicted labels. These prototypes are applied to query features to produce pseudo query labels and formulate a reconstruction constraint during training. By harnessing the contextual information embedded within query features, this approach significantly elevates segmentation performance. Extensive results on two popular benchmark datasets validate the superiority of our model, especially in the challenging 1-shot settings. Under the classic experimental setup, our method surpasses existing state-of-the-arts by 2.64% in 2-way 1-shot setting on the S3DIS dataset. On the ScanNet dataset, the improvements are 7.58% in 2-way 1-shot setting and 6.44% in 3-way 1-shot setting, respectively.